This application contains subject matter which is related to the subject matter of the following applications, each of which is assigned to the same assignee as this application and each of which is hereby incorporated herein by reference in its entirety:
xe2x80x9cMethod To Play Audio And Video Clips Through MPEG Decoders,xe2x80x9d Ser. No. 09/046,283, filed Mar. 23, 1998; and
xe2x80x9cSystem And Method For Merging Multiple Audio Streams,xe2x80x9d Ser. No. 09/273,919, filed Mar. 22, 1999.
The present invention relates generally to digital data decoding systems, and more particularly, to a processing facility capable of receiving multiple concurrent streams of digital audio data and simultaneously outputting therefrom both a mixed digital audio signal and an unmixed digital audio signal.
Full motion video and audio displays based on digital signals have become widely available. While these displays have many advantages, they also often require a massive amount of raw digital data. Because the storage and transmission of digital video and audio signals is central to many applications, and because an uncompressed representation of a video and audio signal requires a large amount of storage, the use of digital compression techniques is vital to this advancing art.
Several international standards for the compression of digital video and audio signals have emerged over the past decade, with more currently under development. These standards apply to algorithms for the transmission and storage of compressed digital data in a variety of applications, including: video-telephony and teleconferencing; high quality digital television transmission on coaxial and fiberoptic networks; as well as broadcast terrestrially and other direct broadcast satellites; and in interactive multimedia products on CD-ROM, digital audio tape, and Winchester disk drives.
Several of these standards involve algorithms based on a common core of compression techniques, e.g., the CCITT (Consultative Committee on International Telegraphy and Telephony) Recommendation H.120, the CCITT Recommendation H.261, and the ISO/IEC MPEG-1 and MPEG-2 Standards. The MPEG algorithms have been developed by the Moving Picture Experts Group (MPEG), part of a joint technical committee of the International Standards Organization (ISO) and the International Electro-technical Commission (IEC). The MPEG Committee has been developing standards for the multiplex, compressed representation of video and associated audio signals.
Briefly summarized, the MPEG-2 standard describes an encoding method that results in substantial bandwidth reduction by a subjective lossy compression followed by a lossless compression. The encoded, compressed digital data is subsequently decompressed and decoded in an MPEG-2 compliant decoder.
Video and audio decoding in accordance with the MPEG-2 standard is described in greater detail in commonly assigned U.S. Pat. No. 5,576,765, entitled xe2x80x9cVideo Decoderxe2x80x9d, which is hereby incorporated herein in its entirety, as well as the above-incorporated, commonly assigned pending application Ser. No. 09/046,283, entitled xe2x80x9cMethod To Play Audio And Video Clips Through MPEG Decodersxe2x80x9d.
Audio/video decoders are typically embodied as general or special purpose processors and memory. Decoders that are used with television sets are often referred to in the industry as set-top box (STB) systems.
Stand-alone audio/video decoders currently used in STB systems generally utilize a dedicated interface to a specific transport chip and/or network interface module (NIM). As the required utility of these STB systems expands, it is becoming desirable to adopt input from various sources in addition to the base network (e.g., cable/satellite) connection.
For example, in addition to a main audio stream, such as a soundtrack for a movie, it may be desirable for the transport stream carrying the program to also contain a secondary audio stream or channel that provides, e.g., additional narrative for disabled viewers who are unable to hear subtle sounds in the original soundtrack. Alternatively, it may be desirable to allow a user to run other functions in the background and request audio queues when certain events have occurred, such as receiving mail, etc. Further, it may be desirable to allow an internet browser or other interactive application to run in the foreground, with primary audio and video programming (e.g., a movie) running in a window on the display. In such cases, it is desirable that the secondary or supplementary audio sound be mixed with the original audio stream so that both can be heard at a single time through a common set of speakers, as opposed to mixing the two streams where only one or the other is heard.
As an additional consideration, in order to accomplish decoding of multiple audio streams, the base audio decoder would conventionally be replicated for as many digital audio streams as output is desired. However, this could be costly and unnecessary in many potential audio mixing applications such as those noted above.
Therefore, in order to establish commercial advantage, there is a need for an integrated device capable of decoding and merging multiple independent digital audio streams using a single audio decode facility. The present invention is directed to meeting this need, as well as others described herein.
In set top boxes, there is an increasing need to be able to play multiple audio streams concurrently by mixing two streams together to a single composite output. As a further example, message overlays can be used when web browsing, while simultaneously watching either a program or listening to other audio content. Service providers are requiring this support for their designs.
Existing solutions have focused on combining the audio streams in the analog domain after decoding and digital to analog conversion. One technique for analog mixing is described in the above-referenced patent application entitled xe2x80x9cSystem And Method For Merging Multiple Audio Streams.xe2x80x9d However, recognized herein is a need for a facility for digitally processing multiple streams of audio data, and for simultaneously outputting of both a mixed digital audio signal and an unmixed digital audio signal, for example, for simultaneous listening and taping, respectively.
Briefly summarized, therefore, the present invention meets these needs by providing a method for processing digital audio data which includes: receiving a first stream of digital audio data and a second stream of digital audio data; outputting the first stream of digital audio data as an unmixed digital audio signal; and digitally mixing the first stream of digital audio data and the second stream of digital audio data and outputting based thereon a mixed digital audio signal, wherein the mixed digital audio signal is output concurrent with output of the unmixed digital audio signal.
In enhanced aspects, the method includes concurrently receiving into an audio decoder the first and second streams of digital audio data through dedicated real time interfaces, wherein the audio decoder decodes at least one of the first stream of digital audio data and the second stream of digital audio data. Further, if digitized at different rates, resampling of the second stream of audio data can occur based on a sample frequency of the first stream of digital audio data. Also, n additional streams of digital audio data, wherein nxe2x89xa71, can be received and digitally mixed for output as the mixed digital audio signal.
Systems, articles of manufacture and program storage devices corresponding to the above-summarized processing facilities are also described and claimed herein.
Advantageously, in one embodiment the present invention provides an ability to send a second stream of digital audio data to an audio decoder for processing by its internal digital signal processor, e.g., using a second xe2x80x9cclip modexe2x80x9d type interface such as that described in the above-incorporated application entitled xe2x80x9cMethod To Play Audio And Video Clips Through MPEG Decoders.xe2x80x9d As a result, the audio decoder can simultaneously receive a stream through either its transport interface or a first clip mode interface and also through a second clip mode interface. One application might be to send pulse code modulated (PCM) uncompressed data through the second interface for resampling and weighted mixing with the audio stream being decoded and played through the primary interface. Note, however, there are no direct restrictions on the type of data that is input through the first or second interface. Any streaming content could be sent that is within the processing power of the available digital signal processor (DSP). This can be extended to other compressed data types, and even non-audio content for DSP processing (e.g., software modem) in parallel with the primary audio stream if there are enough cycles available to handle two real time tasks.
As a further advantage, an audio decoder in accordance with the present invention provides multichannel output. This might entail a front left-right pair, a rear left-right pair, and a center or surround channel. Often, audio applications only require a subset of the possible channels (simple stereo). Since the output channels are under the DSP control, in the case of mixed audio streams one set of stereo channels might be used for the mixed, or composite stream, for example, for direct playback to a user, while the other stereo channel can be used for output of the original unmixed digital audio data, for example, for recording on a VCR or other device. In this manner, message overlays can be heard on the live broadcast, but not recorded for future listening.
To restate, provided herein is an audio device, for example, for an integrated set top box design, that is able to receive multiple concurrent input streams for processing simultaneously into a single mixed digital output signal. Further, the audio device, after mixing, for example, a primary digital input stream with one or more secondary input streams, outputs both the original primary stream and the mixed output stream for concurrent storage and playback by separate devices. The input can be received through a transport and/or clip mode interface, decompression of one or more streams of data can occur, as well as resampling of the secondary stream based on a frequency of the primary stream. All of these functions can be accomplished by the digital signal processor.
Prior solutions are implemented either in the analog domain, which requires additional components and may not provide the mixing granularity or the audio output quality of the present invention, or if in the digital domain require additional component parts to implement, which may make the solution more costly to build and more complex to control. A facility in accordance with the present invention can be readily implemented by one skilled in the art within a single chip set top box controller without the addition of any external components.
Additional features and advantages are realized through the techniques of the present invention. Other embodiments and aspects of the invention are described in detail herein and are considered part of the claimed invention.